Skip to content

boston-lot: metadata review loop - #227

Open
pointcloud-org wants to merge 3 commits into
mainfrom
boston-lot-metadata-review
Open

boston-lot: metadata review loop#227
pointcloud-org wants to merge 3 commits into
mainfrom
boston-lot-metadata-review

Conversation

@pointcloud-org

Copy link
Copy Markdown
Contributor

Exercising the new metadata inventory + explain verb against Boston Lot.

The manifest still has license: UNKNOWN and a TODO description on purpose — that is what we are trying to resolve with LLM-assisted enrichment.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

boston-lot

  • ✅ file existence: all referenced objects found
  • ⚪ pdal_filters: none declared -- nothing to check
  • ⚪ CRS consistency: not applicable (no items_dir)

@pointcloud-org

pointcloud-org commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

📋 Metadata inventory — boston-lot

Everything in the tables below is parsed straight out of the submitted file's LAS public header and VLR records (a few kilobytes, read with a range request — the file itself is never downloaded). Nothing here is inferred or model-written.

What the file says about itself

boston-lot-2021-09-01-CRREL-RSGIS.copc.laz
Points 1,746,342,476
File size 18.5 GiB
Format LAS 1.4, point format 6, COPC
CRS NAD83 / UTM zone 18N
Linear units metre
XY extent 5633 × 6273
Z range -2159.6 – 742.7
Acquiring system empty
Generating software empty
File creation date unset
GPS time range none (0–0)
Extra dimensions 5
VLRs 4
Dimensions and VLRs — boston-lot-2021-09-01-CRREL-RSGIS.copc.laz

Extra dimensions (beyond the standard LAS point format)

Name Type Description
HeightAboveGround double
Synthetic uint8
Overlap uint8
Withheld uint8
KeyPoint uint8

Variable Length Records

# user_id record_id Bytes What it is
0 copc 1 160 COPC info
1 laszip encoded 22204 46 LASzip compression
2 LASF_Projection 2112 615 OGC WKT projection
3 LASF_Spec 4 960 Extra Bytes (extra dimensions) — HeightAboveGround (double), Synthetic (uint8), Overlap (uint8), Withheld (uint8), KeyPoint (uint8)

What the file does not tell us

These are the metadata a reviewer would otherwise assume came from the data. They did not — so if the manifest states them, they came from somewhere else.

  • LAS system_id is empty — the file names no acquiring system
  • LAS generating_software is empty — the file names no creating application
  • no file creation date (creation_year is unset, which PDAL reports as 1) — any acquisition date in the manifest comes from a filename or a human, not from the data
  • GPS time range is 0–0 — the points carry no usable timestamps, so the data cannot date itself
  • no pdal_metadata VLR — the file carries no record of the pipeline that produced it (PDAL only writes this when asked with pdal_metadata=true)
  • declared Z range spans 2902 m (-2159.6 to 742.7) — larger than almost any real survey, so the header bounds are probably set by outlier points rather than terrain; anything derived from them (elevation ranges, hero-image colour ramps) will be skewed until they are filtered

Manifest fields still needing an answer

  • license is UNKNOWN — a real SPDX identifier or proprietary is needed before publication. The model may quote what a source says about licensing, but a human decides it.
  • description is still a placeholder (TODO: write a one-line summary for Boston Lot)
  • no pointcloud_org.acknowledgement
  • no sci:citation or sci:doi — nothing for a user to cite

Reply with @pointcloud-org, please explain … to ask about any of this, or point the bot at an external metadata document and it will read it as part of answering.

@hobu

hobu commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please explain what we actually know about this dataset's provenance, and whether the 2021-09-01 date is supported by anything

@pointcloud-org

pointcloud-org commented Aug 21, 2026

Copy link
Copy Markdown
Contributor Author

🤖 Request received from @hobu (admin access) via comment.

add a description and keywords for this dataset. It is a scan of the Boston Lot Conservation Area in Lebanon, NH (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12) taken in 2021 by USACE CRREL using their "Helipod" — a RIEGL VUX1 mounted on a helicopter. Ground/noise/not-ground classification was done with the PDAL filters.smrf filter.

Working on it…

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

I couldn't tell which command you meant. Supported commands:

  • @pointcloud-org​, please reingest <DATASET_ID> — re-run ingest for one dataset. Opens a pull request for an admin to merge; the ingest happens on merge.
  • @pointcloud-org​, please refresh-stac — rebuild the archive-wide STAC catalog, collections manifest, and the single-file items.parquet.
  • @pointcloud-org​, please check-reachability — check every dataset's external URLs and report anything unreachable.

Dataset ids are directory names under manifests/ (for grouped datasets, just the leaf, e.g. MN_SEDriftless_3_2021).

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

🤖 explain — asked by @hobu

What the file actually establishes

Everything below is from the LAS header/VLRs, which are ground truth:

  • Format and structure: LAS 1.4, point format 6, COPC, 1,746,342,476 points, 18.5 GiB.
  • CRS: NAD83 / UTM zone 18N, linear units metre — so Z values are metres, no compound-CRS unit trap here.
  • Footprint: 5633 × 6273 (metres) in XY.
  • Processing fingerprint: an Extra Bytes VLR carrying HeightAboveGround (double) plus Synthetic, Overlap, Withheld, KeyPoint as uint8. That combination is characteristic of a PDAL-based post-process (HAG computation, classification flags demoted to extra dimensions) — but it names no tool or version.

What the file does not establish — which is nearly all the provenance

  • system_id empty: the file names no acquiring sensor or platform.
  • generating_software empty: the file names no producing application.
  • No pdal_metadata VLR: no record of the pipeline, so the inference above about PDAL is circumstantial, not documented.
  • creation_year unset (PDAL reports 1): no file creation date.
  • GPS time range 0–0: the points carry no usable timestamps. The data cannot date itself at all.

So there is no in-file evidence of who flew it, with what, when, for whom, or under what license.

The 2021-09-01 date

Not supported by anything you have given me. Its only appearance is inside the filename boston-lot-2021-09-01-CRREL-RSGIS.copc.laz. The two header fields that could corroborate an acquisition date are both blank — creation date unset, GPS time 0–0 — so the date is a human's filename convention, not a measurement. Treat it as an unverified claim, and note that even if the string is faithful it could as easily be a processing or delivery date as a flight date.

The CRREL-RSGIS token is in the same position: a filename string, plausibly an organizational initialism, but nothing in the file corroborates any provider. I am not going to expand it into an institution name on that basis.

I also cannot compare any of this against the manifest — manifests/boston-lot/manifest.yaml could not be read at this commit, so I have no submitted claims for license, description, extent.temporal.interval, providers[], or sci:citation to check.

One substantive data concern, independent of provenance

Declared Z spans -2159.6 to 742.7 m, a 2902 m range. For a ~5.6 × 6.3 km footprint that is not terrain; the header bounds are almost certainly set by a handful of outlier points. Anything a user or the site derives from those bounds — elevation range in the collection extent, hero-image colour ramp — will be wrong until the outliers are filtered.

To close this out you would need

  1. The manifest itself (re-fetch at a readable commit).
  2. An external source for the flight window — a survey report, delivery metadata, or a project page from the originating organization — since the file can never supply it.
  3. A stated license from that same source; nothing in the file speaks to licensing, and I won't infer one.
  4. Confirmation of what CRREL-RSGIS denotes, from the submitter or a source document, before it becomes a providers[] entry.

claude-opus-5 · 2205 in / 1323 out tokens

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire
(https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12),
flown in 2021 by the US Army Corps of Engineers Cold Regions Research and
Engineering Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's
"Helipod", flown from a helicopter. Initial ground / noise / not-ground
classification was done with PDAL's filters.smrf.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

I couldn't tell which command you meant. Supported commands:

  • @pointcloud-org​, please reingest <DATASET_ID> — re-run ingest for one dataset. Opens a pull request for an admin to merge; the ingest happens on merge.
  • @pointcloud-org​, please refresh-stac — rebuild the archive-wide STAC catalog, collections manifest, and the single-file items.parquet.
  • @pointcloud-org​, please check-reachability — check every dataset's external URLs and report anything unreachable.

Dataset ids are directory names under manifests/ (for grouped datasets, just the leaf, e.g. MN_SEDriftless_3_2021).

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12), flown in 2021 by the US Army Corps of Engineers Cold Regions Research and
Engineering Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod", flown from a helicopter. Initial ground / noise / not-ground classification was done with PDAL's filters.smrf.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

I couldn't tell which command you meant. Supported commands:

  • @pointcloud-org​, please reingest <DATASET_ID> — re-run ingest for one dataset. Opens a pull request for an admin to merge; the ingest happens on merge.
  • @pointcloud-org​, please refresh-stac — rebuild the archive-wide STAC catalog, collections manifest, and the single-file items.parquet.
  • @pointcloud-org​, please check-reachability — check every dataset's external URLs and report anything unreachable.

Dataset ids are directory names under manifests/ (for grouped datasets, just the leaf, e.g. MN_SEDriftless_3_2021).

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12), flown in 2021 by the US Army Corps of Engineers Cold Regions Research and
Engineering Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod", flown from a helicopter. Initial ground / noise / not-ground classification was done with PDAL's filters.smrf.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

I couldn't tell which command you meant. Supported commands:

  • @pointcloud-org​, please reingest <DATASET_ID> — re-run ingest for one dataset. Opens a pull request for an admin to merge; the ingest happens on merge.
  • @pointcloud-org​, please refresh-stac — rebuild the archive-wide STAC catalog, collections manifest, and the single-file items.parquet.
  • @pointcloud-org​, please check-reachability — check every dataset's external URLs and report anything unreachable.

Dataset ids are directory names under manifests/ (for grouped datasets, just the leaf, e.g. MN_SEDriftless_3_2021).

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire, flown
in 2021 by the US Army Corps of Engineers Cold Regions Research and Engineering
Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod",
flown from a helicopter. Initial ground / noise / not-ground classification was
done with PDAL's filters.smrf.

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12), flown in 2021 by the US Army Corps of Engineers Cold Regions Research and
Engineering Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod", flown from a helicopter. Initial ground / noise / not-ground classification was done with PDAL's filters.smrf.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

🤖 enrich — requested by @hobu

⚠️ The model call did not produce a proposal: model returned no text (stop_reason=max_tokens) — a small max_tokens can be consumed entirely by the thinking block

Nothing was changed.

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12), flown in 2021 by the US Army Corps of Engineers Cold Regions Research and
Engineering Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod", flown from a helicopter. Initial ground / noise / not-ground classification was done with PDAL's filters.smrf.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

🤖 enrich — requested by @hobu

⚠️ The model call did not produce a proposal: model returned no text (stop_reason=max_tokens) — a small max_tokens can be consumed entirely by the thinking block

Nothing was changed.

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12), flown in 2021 by the US Army Corps of Engineers Cold Regions Research and
Engineering Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod", flown from a helicopter. Initial ground / noise / not-ground classification was done with PDAL's filters.smrf.

@hobu

hobu commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add the acquisition and provenance details for this dataset.

It is a scan of the Boston Lot Conservation Area in Lebanon, New Hampshire
(https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12),
flown in 2021 by the US Army Corps of Engineers Cold Regions Research and Engineering
Laboratory (CRREL). The sensor was a RIEGL VUX-1 mounted on CRREL's "Helipod", flown
from a helicopter. Initial ground / noise / not-ground classification was done with
PDAL's filters.smrf.

@hobu

hobu commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add a description and keywords for this dataset. It is a scan of the Boston Lot Conservation Area in Lebanon, NH (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12) taken in 2021 by USACE CRREL using their "Helipod" — a RIEGL VUX1 mounted on a helicopter. Ground/noise/not-ground classification was done with the PDAL filters.smrf filter.

1 similar comment
@hobu

hobu commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

@pointcloud-org, please add a description and keywords for this dataset. It is a scan of the Boston Lot Conservation Area in Lebanon, NH (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12) taken in 2021 by USACE CRREL using their "Helipod" — a RIEGL VUX1 mounted on a helicopter. Ground/noise/not-ground classification was done with the PDAL filters.smrf filter.

@pointcloud-org

Copy link
Copy Markdown
Contributor Author

🤖 enrich — requested by @hobu

Proposing 3 changes to manifests/boston-lot/manifest.yaml. Nothing has been written — this is a suggestion for you to apply, reject, or edit.

Field Basis What and why
description 🗣 you said The what/where/who/why sentence the conventions ask for. Platform, sensor, operator and classification method are all as asserted by the maintainer; the HeightAboveGround dimension is present in the file's Extra Bytes VLR per the extracted inventory. No acquisition purpose was stated, so none is claimed, and no elevation range is quoted because the header Z range (-2159.6 to 742.7 m) is clearly outlier-driven.
keywords 🗣 you said Keeps the existing terms (dropping the generic 'lidar' in favour of 'airborne-lidar'), adds the platform/sensor and processing terms the maintainer supplied, and adds format/CRS terms taken straight from the file inventory (LAS 1.4 COPC, NAD83 / UTM zone 18N, HeightAboveGround extra dimension). All lowercase and hyphenated.
providers 🗣 you said Adds only a description to the existing sole provider entry, stating the acquisition platform the maintainer described. The pointcloud_org.contact block is reproduced verbatim and roles are unchanged (the maintainer did not say who ran the PDAL classification, so no 'processor' role is claimed).
Proposed manifest changes (YAML — click to expand)
description: >-
  Airborne lidar survey of the Boston Lot Conservation Area in Lebanon, New Hampshire, flown in 2021
  by USACE CRREL with their "Helipod" sensor package (a RIEGL VUX1 scanner mounted on a helicopter).
  Points carry ground/noise/not-ground classification produced with PDAL's filters.smrf, plus a
  HeightAboveGround dimension.
keywords:
  - new-hampshire
  - lebanon
  - boston-lot-conservation-area
  - crrel
  - usace
  - airborne-lidar
  - helicopter-lidar
  - helipod
  - riegl-vux1
  - copc
  - las-1-4
  - smrf
  - ground-classification
  - height-above-ground
  - nad83-utm-18n
providers:
  - name: CRREL
    description: >-
      U.S. Army Engineer Research and Development Center, Cold Regions Research and Engineering
      Laboratory — acquired the survey with its helicopter-mounted "Helipod" (RIEGL VUX1) lidar
      system.
    roles:
      - producer
    url: https://www.erdc.usace.army.mil/Locations/CRREL/
    pointcloud_org:
      contact:
        name: Howard Butler
        email: howard@hobu.co
        github_owner: hobu

On licensing

The model cannot propose license. This is what it found; the decision is yours.

No licensing statement was found in any supplied document. The fetched City of Lebanon facilities page (https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12) describes the conservation area only and says nothing about the lidar data; its footer links a generic "Copyright Notices" page that was not fetched and would in any case cover the city's website, not a USACE CRREL dataset. The manifest's license: UNKNOWN and its TODO should stand. What would settle it: a CRREL/ERDC release or distribution statement for this specific acquisition (or the data-use terms under which hobu received the file) — that document would also fix pointcloud_org.acknowledgement, which is why no acknowledgement is proposed here.

Notes. No acquisition date proposal: the file has no creation date and GPS time range is 0-0, so the existing 2021-09-01 interval rests solely on the filename. The maintainer confirmed the year 2021 but not the day, so the current interval is left untouched; a CRREL flight/acquisition report would settle the exact window. No sci:doi, sci:citation, sci:publications or publication_date proposed for the same reason — no source. The fetched document contained no instruction-like text directed at me; it appears to be ordinary municipal web content, and I used it only as background (it was not needed to support any proposal, so nothing is marked 'cited').

Documents read: https://lebanonnh.gov/facilities/facility/details/Boston-Lot-Conservation-Area-12

claude-opus-5 · 6086 in / 2286 out tokens

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants